Update integer advanced indexing for array API 2024.12 spec by ndgrigorian · Pull Request #2032 · IntelPython/dpctl

ndgrigorian · 2025-03-26T23:14:41Z

The array API specification now requires advanced integer indexing (i.e., with arrays) to promote Python integer scalars to arrays when mixed with integer arrays.

This PR implements this behavior, promoting any integer indices that occur in a sequence with array indices of dimension 1 or greater.

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
Have you added documentation for your changes, if necessary?
Have you added your changes to the changelog?
If this PR is a work in progress, are you opening the PR as a draft?

github-actions · 2025-03-26T23:50:51Z

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

github-actions · 2025-03-26T23:57:12Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_75 ran successfully.
Passed: 894
Failed: 2
Skipped: 154

coveralls · 2025-03-26T23:59:37Z

coverage: 86.392% (+0.01%) from 86.381%
when pulling 2fdac4c on update-indexing-for-array-api-2024
into f8c751b on master.

github-actions · 2025-03-31T17:19:55Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_78 ran successfully.
Passed: 894
Failed: 2
Skipped: 154

dpctl/tensor/_copy_utils.py

dpctl/tensor/_slicing.pxi

dpctl/tensor/_usmarray.pyx

dpctl/tensor/_copy_utils.py

github-actions · 2025-04-02T07:31:13Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_104 ran successfully.
Passed: 894
Failed: 2
Skipped: 154

github-actions · 2025-04-03T02:42:11Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_109 ran successfully.
Passed: 894
Failed: 2
Skipped: 154

github-actions · 2025-04-03T06:13:41Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_110 ran successfully.
Passed: 894
Failed: 2
Skipped: 154

array API 2024 spec requires advanced indexing with integer arrays, and when advanced indexing behavior is triggered, integral scalars are to be converted to arrays and broadcast with the other indices

this fixes cases like `x[..., arr1, int1, arr2, int2, :, int3]`, the final int will be treated as an integer instead of an array, and `arr1` through `int2` will be treated as arrays

since the index is on the host, we can raise like we normally would, which follows NumPy behavior

Reduces code duplication

Typos from advanced indexing updates

Test for the case where an basic integral index appears between two integral arrays, followed by a basic index, and then followed by `:`

ndgrigorian · 2025-04-08T01:35:31Z

@antonwolfy
a couple of new tests added, I think PR is ready for another review pass

github-actions · 2025-04-08T02:16:55Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_116 ran successfully.
Passed: 894
Failed: 2
Skipped: 154

dpctl/tensor/_slicing.pxi

dpctl/tensor/_copy_utils.py

dpctl/tests/test_usm_ndarray_indexing.py

0D array indices are treated as Python scalars and moved to the host when not part of other, consecutive array indices This means that 0D arrays also can't start a series of consecutive advanced indices

github-actions · 2025-04-09T04:10:08Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_123 ran successfully.
Passed: 912
Failed: 2
Skipped: 154

github-actions · 2025-04-09T06:10:16Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_126 ran successfully.
Passed: 913
Failed: 1
Skipped: 154

dpctl/tensor/_slicing.pxi

…en not among consecutive integer arrays

github-actions · 2025-04-09T17:38:37Z

Array API standard conformance tests for dpctl=0.20.0dev0=py310h93fe807_128 ran successfully.
Passed: 912
Failed: 2
Skipped: 154

antonwolfy

Thank you @ndgrigorian, LGTM

ndgrigorian requested review from antonwolfy, vlad-perevezentsev and vtavana as code owners March 26, 2025 23:14

ndgrigorian marked this pull request as draft March 26, 2025 23:15

ndgrigorian force-pushed the update-indexing-for-array-api-2024 branch from 619dbfd to a02b5e5 Compare March 31, 2025 16:36

ndgrigorian marked this pull request as ready for review March 31, 2025 23:23

antonwolfy reviewed Apr 1, 2025

View reviewed changes

ndgrigorian force-pushed the update-indexing-for-array-api-2024 branch from a02b5e5 to 71ca729 Compare April 2, 2025 06:49

ndgrigorian force-pushed the update-indexing-for-array-api-2024 branch from 71ca729 to d426408 Compare April 3, 2025 01:57

ndgrigorian requested a review from antonwolfy April 3, 2025 05:53

ndgrigorian added 9 commits April 7, 2025 18:34

Update integer advanced indexing for array API 2024

87fa69a

array API 2024 spec requires advanced indexing with integer arrays, and when advanced indexing behavior is triggered, integral scalars are to be converted to arrays and broadcast with the other indices

only treat integers as arrays when arrays have been consecutive

47639da

this fixes cases like `x[..., arr1, int1, arr2, int2, :, int3]`, the final int will be treated as an integer instead of an array, and `arr1` through `int2` will be treated as arrays

raise for OOB indices even if they will be converted to arrays

b18fe9d

since the index is on the host, we can raise like we normally would, which follows NumPy behavior

Add common utilities for _put_multi_index and _take_multi_index

9a7705e

Reduces code duplication

Remove redundant IndexError in _slicing.pxi

25fb1a2

Fix over-indentation

14a55d0

Typos from advanced indexing updates

Add tests for uncovered branches in advanced indexing

72f8938

Add another test for advanced indexing

b38478b

Test for the case where an basic integral index appears between two integral arrays, followed by a basic index, and then followed by `:`

Add another test for advanced indexing errors

d873f35

ndgrigorian force-pushed the update-indexing-for-array-api-2024 branch from 12e2121 to d873f35 Compare April 8, 2025 01:34

antonwolfy reviewed Apr 8, 2025

View reviewed changes

dpctl/tensor/_slicing.pxi Show resolved Hide resolved

dpctl/tensor/_copy_utils.py Show resolved Hide resolved

dpctl/tests/test_usm_ndarray_indexing.py Show resolved Hide resolved

dpctl/tests/test_usm_ndarray_indexing.py Show resolved Hide resolved

Partial reversion of advanced indexing changes

0784d70

0D array indices are treated as Python scalars and moved to the host when not part of other, consecutive array indices This means that 0D arrays also can't start a series of consecutive advanced indices

ndgrigorian requested a review from antonwolfy April 9, 2025 03:27

Merge branch 'master' into update-indexing-for-array-api-2024

682b598

antonwolfy reviewed Apr 9, 2025

View reviewed changes

dpctl/tensor/_slicing.pxi Outdated Show resolved Hide resolved

ndgrigorian added 2 commits April 9, 2025 09:03

Use operator.index in _slicing.pxi

87db03a

Adds a test that 0D integral indices are treated as Python scalars wh…

2fdac4c

…en not among consecutive integer arrays

ndgrigorian requested a review from antonwolfy April 9, 2025 16:56

antonwolfy approved these changes Apr 9, 2025

View reviewed changes

ndgrigorian merged commit 4f63b18 into master Apr 9, 2025
72 of 73 checks passed

ndgrigorian deleted the update-indexing-for-array-api-2024 branch April 9, 2025 20:33

Conversation

ndgrigorian commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 26, 2025

Uh oh!

coveralls commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 31, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 2, 2025

Uh oh!

github-actions bot commented Apr 3, 2025

Uh oh!

github-actions bot commented Apr 3, 2025

Uh oh!

ndgrigorian commented Apr 8, 2025

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 9, 2025

Uh oh!

github-actions bot commented Apr 9, 2025

Uh oh!

Uh oh!

github-actions bot commented Apr 9, 2025

Uh oh!

antonwolfy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ndgrigorian commented Mar 26, 2025 •

edited

Loading

github-actions bot commented Mar 26, 2025 •

edited

Loading

coveralls commented Mar 26, 2025 •

edited

Loading